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PREFACE 

The investigation which is the subject of this document was undertaken in sup- 
port of the Foreign Commodity Production Forecasting project of the Agricul- 
ture and Resources Inventory Surveys Through Aerospace Remote Sensing program. 
Under Contract NAS 9-15800, scientists of Lockheed Engineering and Management 
Services Company, Inc., evaluated the results which are reported for the Earth 
Observations Division, Space and Life Sciences Directorate, of the National 
Aeronautics and Space Administration, Lyndon B. Johnson Space Center. 
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1. INTRODUCTION 


The purpose of the U.S. Corn and Soybeans Exploratory Experiment — Classifi- 
cation Procedures Verification Test was to evaluate the performance of the 
adapted Large Area Crop Inventory Experiment (LACIE) Transition Year (TY) 
classification procedure for corn and soybeans. See reference 1 for a 
discussion of the procedure used in this test. In this test, 25 segments 
selected from four agrophysical units (APU's) were processed by three groups 
of analysts. Analysis of variance techniques were used to determine the 
factors which were Important to the quality of the classifications per- 
formed. The factors evaluated were group effects and APU effects. The 
classification results were evaluated to determine the effectiveness of the 
procedure in producing corn and soybeans proportion estimates. 
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2. FACTORS AFFECTING THE QUALITY OF THE CLASSIFICATIONS 

The segments used in this test were from APU's 14, 24, 26, and 28 located in 
Missouri, Iowa, Illinois, and Indiana. Because APU 24 had a small number of 
segments and APU's 14 and 24 were reasonably similar, APU's 14 and 24 were 
merged and designated APU 14 for evaluation purposes. 

Three groups of analysts processed the segments. Group I processed 19 of the 
segments, whereas groups II and III each processed 18 segments. The alloca- 
tion of the segments among the groups and APU's is shown in table 1. The 
linear model and related assumptions used in the analyses of variance are des- 
cribed in reference 2. 

The following measures of classification quality were used in the analyses of 
variance: 

a. Proportion estimation error 

b. Percentage of picture elements (pixels) correctly classified 

c. Reduction in the expected proportion estimate variance if a bias correc- 
tion were applied to the classification results 

d. Analyst dot labeling accuracy 

The factors were tested for their effects in the following order: first, 

interaction between groups and APU's; second, group effects; and, third, APU 
effects. If a significant result was obtained at one stage, it was impossible 
to test for significant results at a later stage. 

Table 2 shows the average proportion estimation error and average absolute 
proportion error for corn and soybeans by group and by APU. Significant dif- 
ferences are indicated by numbers in parentheses following the values. No 
significant effects were found in the results for corn. For soybeans, a sig- 
nificant difference in the proportion errors was found between groups II and 
III. The absolute proportion error was significantly different for APU 14. 
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TABLE: 1.- DISTRIBUTION OF SEGMENTS BY GROUP AND BY APU 


[Parentheses Indicate processed data which were not used 
In the analyses of variance] 


Segment 


APU 

number 

Group I 

Group II 

Group : 

14 

135 

X 


X 


202 

(X) 

(X) 

(X) 


864 


X 

X 


865 

X 


X 


877 

X 

X 



880 


X 

X 


881 

X 

X 

(X) 


882 

(X) 

(X) 

(X) 

25 

107 

X 

X 



141 

V 

/\ 


A 


144 


X 

X 


205 

X 

X 



800 

(X) 

(X) 



807 


X 

X 


809 

X 


X 

28 

123 

X 

X 



127 

(X) 

(X) 

(X) 


133 


X 

X 


832 

X 


X 


837 

(X) 

(X) 

(X) 


842 

X 

X 



843 

(X) 

(X) 



852 

X 


X 


853 

(X) 


(X) 


860 


X 

X 
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TABLE 2.- PROPORTION ESTIMATION ERRORS 

[Significant differences are indicated by number 
in parentheses following the values] 



Corn 

Soybeans 

Average 
error, % 

Average 
absolute 
error, % 

Average 
error, % 

Average 
absolute 
error, % 

Group I 

-6.3 

7.4 

-6.6 

7.4 

Group II 

-3.1 

8.x 

-9.0(1) 

9.0 

Group III 

-4.8 

7.1 

-4.0(1) 

7.0 

APU 14 

-5.8 

7.4 

-2.3 

4.5(2)(3) 

APU 25 

-3.6 

5.9 

-7.3 

9.0(2) 

APU 28 

-4.8 

9.3 

-9.9 

9.9(3) 

Overall 

-4.7 

7.5 

-6.5 

7.8 


The results for the percentage of pixels correctly classified are shown in 
table 3. An Interaction between groups and APU's for the percentage of cor- 
rect classification (PCC) for class "other" made it impossible to determine 

group and APU effects for the PCC for "other." The only significant result 

was a group effect for the PCC for corn, where the group III result was sig- 
nificantly different from the group I and II results. 

The results of reductions In variance are shown in table 4, In analyzing the 
results for corn, a significant interaction between groups and APU's made it 
impossible to test for group and APU effects Individually. There were no sig- 
nificant effects for soybeans. 

Tables 5 and 6 show the dot labeling accuracy for type 1 and type 2 dots. 

There were group effects for the type 1 dot labeling accuracy for corn and for 
the overall category. In both cases, group III was significantly different 

from groups I and II. A significant APU effect was shown for the labeling 

accuracy for class "other" in both the type I and type 2 dots. In both cases, 
APU 14 was significantly different from APU's 25 and 28. 

In summary, the observed group effects involved dot labeling accuracy and PCC 
for corn. In both cases, group III was consistently less accurate than 
groups I and II. Since all three groups were given the same training and were 
to follow the same procedures, it would appear that there was some misunder- 
standing of the procedure for corn by group III. 

The observed APU effects involved dot labeling accuracy and proportion estima- 
tion error for soybeans. In both cases, APU 14 had less accurate results than 
APU's 25 and 28. It appears that dot labeling for soybeans is more difficult 
in APU 14. It is interesting to note that, although the dot labeling for 
type 1 dots showed a significant difference, the PCC for the classifications 
based on these dots did not show a significant difference. 


TABLE 3.- PERCENTAGE OF PIXELS CORRECTLY CLASSIFIED 

[Significant differences are Indicated by number 
in parentheses following the values] 


Group I 
Group II 
Group III 
14 
25 
28 

Overall 


Corn PCC 

73.2(1) 
75.6(2) 
62.6(1) (2) 

77.8 

69.9 
63.6 


Soybeans PCC 


64.2 

52.5 

53.9 

59.9 

49.8 

60.9 


"Other" PCC Overall PCC 


TABLE 4.- PERCENTAGE OF REDUCTION IN VARIANCE EXPECTED IF BIAS 
CORRECTION IS PERFORMED ON CLASSIFICATION RESULTS 












TABLE 5.- TYPE 1 DOT LABELING ACCURACY 


[PCL * percentage of dots correctly labeled; significant differences 
are indicated by the number in parentheses following the values] 



Corn PCL 

Soybeans PCL 

"Other" PCL 

Overall PCL 

Group I 

88.3(1) 

79.9 

89.5 

86.8(3) 

Group II 

89.2(2) 

76.2 

88.3 

86.8(4) 

Group III 

67.0(1 ) (2) 

66.1 

85.8 

77.8(3) (4) 

APU 14 

83.5 

83.3 

76.9(5) (6) 

83.5 

APU 25 

85.9 

65.1 

89.6(5) 

82.7 

APU 28 

75.1 

73.8 

97.1(6) 

85.1 

Overall 

81.5 

74.1 

87.9 

83.8 


TABLE 6.- TYPE 2 DOT LABELING ACCURACY 

[PCL = percentage of dots correctly labeled; significant differences 
are indicated by the number in parentheses following the values] 



Corn PCL 

Soybeans PCL 

"Other" PCL 

Overall PCL 

Group I 

66.9 

70.4 

85.9 

74.9 

Group II 

70.5 

60.6 

86.5 

74.3 

Group III 

64.5 

61.1 

80.7 

70.9 

APU 14 

70.8 

72.8 

76 .6(1 ) (2) 

73.6 

APU 25 

70.7 

61.8 

89.3(1) 

76.3 

APU 28 

60.5 

57.5 

87.2(2) 

70.3 

Overal 1 

67.3 

64.0 

84.4 

73.4 




3. CLASSIFICATION PROCEDURE EVALUATION 


In order to determine the effectiveness of the classification procedure in 
producing proportion estimates, the various stages in the classification pro- 
cedure must be investigated. One way of doing this is to calculate proportion 
estimates based only on the information available at a particular stage. By 
comparing the accuracy at the different stages, one can determine which steps 
are necessary and which steps are not. 

The classification procedure consists of the following steps: 

a. Two sets of dots are labeled as corn, soybeans, or "other" by the analyst. 

b. Using one set of analyst-labeled (type 1) dots as seed pixels, all pixels 
in the segment are grouped into clusters on the basis of their spectral 
values. 

c. Each of the clusters is labeled as corn, soybeans, or "other" by the 
analyst-labeled type 1 dot closest to the mean of the cluster. 

d. On the basis of the means and variances for each cluster, -yery pixel in 
the segment is classified as corn, soybeans, or "other.' 1 

e. Using the second set of analyst-labeled (type 2) dots as a random sample 
of the segment, the proportions based on the classification are corrected 
for any bias introduced by the classification process. 

Proportion estimates can be calculated at the following four stages in the 
classification procedure: 

a. At the dot labeling stage, the type 2 dots can be aggregated on the basis 
of their labels to determine a proportion. 

b. At the clustering stage, a proportion can be determined by aggregating the 
pixels in a cluster on the basis of the label assigned to the cluster. 

c. At the classification stage, a proportion can be determined by aggregating 
the pixels on the basis of the labels assigned by the classifier. 

d. At the bias-correction stage, the final estimate produced by the procedure 
can be used. 


The set of classifications used in this evaluation is listed in table 1. For 
the purposes of evaluating the classification process, five of the classifica' 
tlons were not used: 882 and 127 by group I; 881 by group II; 837 and 860 by 
group III. Eliminating these classifications resulted in each segment being 
represented twice by two different groups. Groups I and II were represented 
17 times each, whereas group III was represented 16 times. 

Although it is possible to determine a proportion at the clustering stage, 
clustering proportions are not presented. The cluster-based proportions are 
not included because the cluster and classification proportions are essen- 
tially identical. Figure 1 shows the classification proportions P(CLS) as a 
function of the cluster proportions P(CLU) for the segments involved in this 
evaluation. The linear regressions shown in the figure indicate an almost 
perfect correlation between the two proportion estimates (R 2 = 0.S9907). 
Therefore, proportion estimates are calculated for the type 2 dots, clas- 
sification, and bias-correction stages. 

Figure 2 shows the errors in the proportion estimates as a function of the 
true proportion. The mean error, standard deviation, and mean square error 
for each estimator are presented in table 7 (page 3-7). The mean error is a 
measure of the bias in the estimator. The standard deviation is a measure of 
the estimator's variability. The mean square error is an indication of the 
overall performance of the estimator. 

The mean error for corn was negative at the dot labeling and bias-correction 
stages and positive at the machine classification stage. The mean square 
errors were nearly the same at the dot labeling and bias-correction stages. 
This indicates that the machine processing did not improve the proportion 
estimate. The type 2 dots produced as good an estimate by themselves as 
when they were used to establish a bias-correction factor for the machine 
classification. 
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Figure 2.- Concluded. 



The mean error for soybeans was negative at all three stages. At the machine 
classification stage, the mean error and standard deviation were actually less 
than at the dot labeling and bias-correction stages. As in the corn esti- 
mates, the type 2 dots produced as good an estimate alone as when they were 
used to establish a bias-correction factor for the machine classification. 

Table 8 shows the relative improvement in the estimate at one stage over that 
of the previous stage. The percentage of classifications improved and the 
mean improvement are indicated. The improvement is defined as the reduction 
in the absolute value of the error in going from one stage to the next. (For 
example, if the error at one stage was 5 percent and the error at a subsequent 
stage was -3 percent, then the improvement would be 2 percent.) These results 
are consistent with the results presented in table 7. If one uses the square 
root of the mean square error as a measure of the overall error at a given 
stage, the improvements are, in all cases, less than 20 percent of this value. 
Therefore, the machine processing has made a minimal change in proportion 
estimation accuracy. 


The sources of the proportion errors can be separated into input errors in the 
form of mi t labeled dots and errors in the proportion estimation procedure 
itself. In order to determine the amount of error introduced by the propor- 
tion estimation procedure, classifications were performed using ground-truth 
dot labels as input. The errors in these classifications are caused by the 
procedure alone. The results of these ground-truth-based classifications are 
shown in figure 3. The mean errors, standard deviations, and mean square 
errors are presented in table 9. Table 10 shows the improvements at the vari- 
ous stages in the process. 

For both corn and soybeans, about 60 percent of the classifications had better 
bias-correction estimates than dot labeling estimates. However, the mean 
improvement was less than 1 percent. The bias-correction and dot labeling 
estimates are not significantly biased. Although there was an improvement 
between the dot labeling estimate and the bias-correction estimate, the 
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TABLE 7.- U.S. CORN AND SOYBEANS EXPLORATORY 
EXPERIMENT _ CLASSIFICATION ERRORS 



Corn 

Soybeans 

Source of 
classification 

Mean 

error 

Standard 

deviation 

Mean 

square 

error 

Mean 

error 

Standard 

deviation 

Mean 

square 

error 

Type 2 dots as 
random sample 

-3.71 

8.15 

78.9 

-5.76 

7.34 

85.9 

Machine classification 

2.43 

10.00 

103.8 

-4.67 

6.33 

61.0 

Bias-corrected 
machine classification 

-3.83 

7.74 

73.4 

-5.80 

6.91 

80.4 


TABLE 8.- U.S. CORN AND SOYBEANS EXPLORATORY 
EXPER IMENT _ CLASS IF ICAT ION IMPROVEMENT 


Classification 
sources compared 

Corn 

Soybeans 

Processing 
improved, % 

Mean 

improvement 

Processing 
improved, % 

Mean 

improvement 

Machine classification 
vs. type 2 dots 

49 

-0.34 

55 

1.36 

Bias correction vs. 
machine classification 

55 

0.91 

43 

-1.24 

Bias correction vs. 
type 2 dots 

51 

0.57 

45 

0.12 
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Figure 3.- Proportion estimates using ground-truth labeling as input. 




TABLE 9.- U.S. CORN AND SOYBEANS EXPLORATORY EXPERIMENT - CLASSIFICATION 
ERRORS USING GROUND-TRUTH LABELS AS INPUT 



Corn 

Soybeans 

Source of 
classification 

Mean 

error 

Standard 

deviation 

Mean 

square 

error 

Mean 

error 

Standard 

deviation 

Mean 

square 

error 

Type 2 dots as 
random sample 

1.55 

5.19 

28.3 

1.00 

4.14 

17.5 

Machine classification 

8.21 

8.98 

144.7 

-2.28 

5.63 

35.6 

Bias-corrected 
machine classification 

1.00 

4.07 

17.0 

0.47 
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TABLE 10.- U.S. CORN AND SOYBEANS EXPLORATORY EXPERIMENT - CLASSIFICATION 
IMPROVEMENT USING GROUND-TRUTH LABELS AS INPUT 


Classification 
sources compared 

Corn 



Soybeans 

Processing 
improved, % 

Mean 

improvement 

Processing 
improved, % 

Mean 

improvement 

Machine classification 
vs. type 2 dots 

20 

-5.05 

36 

-1.26 

Bias correction vs. 
machine classification 

76 

5.70 

76 

2.24 

Bias correction vs. 
type 2 dots 

60 

0.65 

64 

0.98 
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improvement is not great enough to warrant the effort involved in performing 
the machine classification. 

The most interesting feature of the ground-truth-based classification results 
is the large mean error in the machine classification proportions for corn. 

The plot in figure 3 shows that the error increases with increased true pro- 
portion. In fact, the mean square error of 144.7 (table 9) is larger than the 
mean square error of 103.8 for the analyst-based machine classification 
results (table 7). This indicates a serious problem with the procedure, since 
one would expect the results to Improve or remain the same when true labels 
are substituted for analyst labels. 

A possible source for the bias could be that the type 1 dots, used as input 
for the classification, are not representative of the entire segment. In 
order to determine if the type 1 dots are representative of the segment as a 
whole, a proportion estimate can be calculated using the type 1 dots as a ran- 
dom sample of the segment. If the type 1 dots are representative of the seg- 
ment, the estimate should be unbiai&t . Figure 4 shows the proportion estima- 
tion error for the type 1 dots. As one might expect, the corn estimate has an 
8.48-percent positive bias. This is very close to the bias of 8.21 percent in 
the classification estimate. The type 1 dot estimate shows the same trend as 
the classification estimate. Therefore, the type 1 dots are not representa- 
tive of the segment, which is responsible for the bias in the classification 
results. 

Th? question to consider now is: Why are the type 1 dots a biased sample of 
the segment? These dots are a set taken from a random grid; thus, the loca- 
tion should not produce a bias. One restriction was placed on the dots: that 
a dot which falls on a field boundary is not used. In this particular test, 
type I dots were used only if they were more than one-half pixel away from a 
field boundary. If the proportion is calculated using all of those pixels 
which meet the purity criterion and this estimate is biased with respect to 
the true proportion, then the purity restriction on the type 1 dots is the 
source of the observed bias. Figure 5 shows errors in the proportions based 





% 


on all pure pixels in the segment as a function of the true proportion. The 
proportion errors for corn show the same trend to greater error with increased 
proportion, as seen in the type 1 dot proportion and classification results. 
The mean error f or corn is 7.61 percent, which is consistent with the errors 
observed for the type 1 dot and classification estimates. 

The conclusion from chis analysis would be that the type 1 dots are more 
representative of the pure pixels in the scene than of the entire scene. 

Since the pure pixels are a biased sample of the segment, the proportions 
based on the cype 1 dots and on the classification will also be biased. One 
w? M of verifying this conclusion is to compare the proportion estimates with 
the ground-truth proportions based on pur;e pixels. If the mean error, stand- 
ard deviation, and mean square error are less when the pure pixel ground-truth 
proportion is used rather than the entire scene ground-truth proportion, then 
the proportions are more representative of the pure pixels than of the entire 
scene. Figure 6 shows the results of these comparisons. The corn estimates 
do not show the large positive bias evident when the entire scene proportion 
is used as the true proportion. The mean errors, standard deviations, and 
mean square errors corresponding to figure 6 are presented in table 11. The 
mean errors for the corn estimates are reduced from more than 8 percent to 
less than 1 percent. There was a slight reduction in the standard deviation. 
The mean square error was reduced by 50 percent or more. The results for soy- 
beans were not as straightforward as those for corn. Although the mean square 
error for the type 1 dots decreased slightly when pure pixel proportions were 
used, the mean square error for the classification actually increased. These 
changes are not significant because the pure pixel and entire scene ground- 
truth proportions were close. 

The bias and about one-half of the variability in the proportion estimates are 
the result of analyst dot labeling errors. A summary of the analyst dot 
labeling accuracy is shown in tables 12 and 13. The overall accuracy for 
type 1 dot labeling was 86 percent, whereas the accuracy for type 2 dot label- 
ing was 75 percent. This is probably a consequence of the fact that all of 
the type 1 dots were pure, whereas type 2 dots could be impure. One can 
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Figure 6.- Concluded. 



TABLE 11.- EFFECT OF USING PURE PIXEL GROUND-TRUTH PROPORTIONS 

ON CLASSIFICATION ERRORS 


Crop 

Source of 
classification 
estimate 

Source of 
ground-truth 
proportion 

r 

Mean 

error 

Standard 

deviation 

Mean 

square 

error 

Corn 

Type 1 dots as 
random sample 

Entire scene 

8.48 

13.19 

238.9 

Pure pixels 

.93 

10.69 

110.6 

Machine 

classification 

Entire scene 

8.21 

8.98 

144.7 

Pure pixels 

.66 

7.32 

51.9 

Soybeans 

Type 1 dots as 
random sample 

Entire scene 

.96 

8.38 

68.4 

Pure pixels 

-1.18 

6.97 

48.0 

Machine 

classification 

Entire scene 

-2.28 

5.63 

35.6 

Pure pixels 

-4.41 

4.93 

42.8 
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TABLE 12.- DOT LABELING ACCURACY FOR TYPE 1 DOTS 


Crop 

Dots 

labeled 

corn 

Dots 

labeled 

soybeans 

Dots 

labeled 

"other" 

Dots 

correctly 
labeled, % 

Corn 

647 

34 

71 

86 

Soybeans 

54 

392 

52 

79 

"Other" : 





Wheat 

3 

0 

23 

88 

Oats 

1 

0 

8 

89 

Grass 

0 

1 

7 

88 

Hay 

3 

2 

40 

89 

Pasture 

7 

1 

138 

95 

Trees 

6 

1 

142 

95 

Clover 

0 

0 

9 

100 

Vegetable 

0 

0 

2 

100 

Water 

0 

0 

14 

100 

Nonagriculture 

1 

3 

41 

91 

Homestead 

1 

0 

27 

96 

Idle 

3 

2 

35 

88 

Total "other" 

25 

10 

486 

93 
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TABLE 13.- DOT LABELING ACCURACY FOR TYPE 2 DOTS 


Crop 

Dots 

labeled 

corn 

Dots 

labeled 

soybeans 

Dots 

labeled 

"other" 

Dots 

correctly 
labeled, % 

Corn 

1598 

124 

456 

73 

Soybeans 

231 

1014 

341 

64 

"Other": 





Wheat 

11 

11 

93 

81 

Oats 

14 

3 

64 

79 

Grass 

6 

3 

22 

71 

Hay 

6 

8 

124 

90 

Pasture 

47 

18 

421 

87 

Trees 

18 

8 

343 

93 

Clover 

4 

2 

5 

45 

Vegetable 

0 

0 

9 

100 

Water 

2 

0 

35 

95 

Nonagriculture 

12 

10 

131 

86 

Homestead 

7 

6 

95 

88 

Idle 

21 

13 

119 

78 

Total "other" 

148 

82 

1461 

86 




explain the fact that the soybean proportion estimates based on classification 
results were better than those based on the type 2 dots when analyst labels 
were used. Although the classification estimates are usually less accurate, 
the better labeling for the type'l dots was enough to improve the classifica- 
tion results. In looking at the confusion between the categories (corn, soy- 
beans, and "other"), it appears that there is greater confusion between corn 
and "other" than between corn and soybeans. 

In order to determine how well the clustering algorithm is working in separat- 
ing the crop of interest from a noncrop, the cluster purities were calculated 
for corn and for soybeans. Histograms of cluster purity are shown for corn 
and soybeans in figures 7 and 8. The number of clusters with given crop pro- 
portions is plotted as a function of the crop proportion. Ideally, these his- 
tograms should show two maxima (at 0 percent and 100 percent) representing 
pure noncrop and crop clusters. The histogram should be zero at the center. 

In the figures, one does see the expected two maxima with a minimum of approx- 
imately 50 percent. The crop maximum is fairly broad, but it appears that the 
clustering algorithm is separating crop and noncrop pixels to a certain 
extent. 



Number of clusters 



0.00 20.00 40 


80.00 80.00 100.00 


Soybeans proportion in cluster, % 


Figure 8.- U.S. Corn and Soybeans Exploratory Experiment 
histogram of cluster purity for soybeans. 
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4. SUMMARY OF RESULTS 


Based on the studies presented in this document, the following conclusions can 

be reached: 

a. The proportion estimates for corn had a bias of -4 percent with a standard 
deviation of 8 percent. 

b. The proportion estimates for soybeans had a bias of -6 percent with a 
standard deviation of 7 percent. 

c. The bias and about one-half the standard deviation for both corn and 
soybeans were the result of dot labeling errors. 

d. Proportion estimates based on the type 2 dots as a random sample are as 
good as the final bias-corrected results. 

e. The machine classification results are identical to the machine clustering 
results. 

f. The large bias observed in the classification proportions for corn (when 
true labels are used) is caused by bias in the type 1 dots used as input 
to the classification procedure. 

g. The bias in the type 1 dots was present because the type 1 dots were 
required to be pure. 

h. Although the three groups used to process the segments were given identi- 
cal training and used identical procedures, one group had significantly 
different dot labeling accuracy. 

i. It is more difficult to label "other" dots in APU 14 than it is in 
APU's 25 and 28. 


5. RECOMMENDATIONS 


Dot labeling errors are the greatest soiirce of error in the proportion esti- 
mates. If the quality of the proportion estimates is to be improved, the cur- 
rent dot labeling techniques need to be improved or an alternative for dot 
labeling found. 

Since the machine processing used in this test does not significantly improve 
the accuracy of the corn and soybeans proportion estimates, the proportion 
estimates can be made using the labeled dots as a random sample of the 
segment. Alternatives to the machine processing technique used in this test 
should be investigated to see if a more effective technique can be found. 

Since the maximum likelihood classification results are identical to the 
results using labeled clusters, it is not necessary to perform the maximum 
likelihood classification. The proportion estimates based on the clustering 
results should be bias corrected using a random dot set so that the kind of 
bias reflected in the corn proportion estimates can be reduced. 
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